Modifying socket.setdefaulttimeout is not thread-safe as it affects the entire process. In the FastAPI parent process, concurrent requests could interfere with each other's timeout settings. A safer, thread-safe alternative for a reachability check is to use socket.create_connection((host, 443), timeout=timeout), which respects the timeout without mutating global state and supports both IPv4 and IPv6 addresses.

p.relative_to(snap).as_posix() is a relatively expensive path operation and is currently called twice for every file found by rglob. To improve efficiency, compute this value once and reuse the result.

                        matches = sorted(
                            rel
                            for p in snap.rglob("*.gguf")
                            if "mmproj" not in p.name.lower()
                            and (rel := p.relative_to(snap).as_posix())
                            and boundary.search(rel.lower())
                        )

The DNS probe logic here is duplicated from _probe_dns_dead in core/inference/llama_cpp.py. You should centralize this logical check into a single helper function and reuse it across the codebase to ensure consistency and simplify maintenance.

The _env_offline helper is duplicated in both utils/models/model_config.py and utils/transformers_version.py. Since model_config.py already imports from transformers_version.py, you should centralize this logic in transformers_version.py and import it here to improve maintainability and avoid logic drift.

Detect cached LoRA adapters while offline

When HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE is set for a cached remote LoRA repo, this skips the only path that flips is_lora to true; from_identifier() then falls through as if the adapter repo were a full base model, so the later LoRA block never downloads/reads cached adapter_config.json and the inference worker receives is_lora=False. This regresses offline loading of already-cached HF LoRA adapters; the offline path should still check the local HF cache for adapter_config.json (or otherwise honor cached adapter metadata) instead of bypassing detection entirely.

Useful? React with 👍 / 👎.

Bound DNS probes with a real timeout

When a user's resolver is blackholed or very slow, this new probe can block the FastAPI load path well beyond the intended 2 seconds because socket.setdefaulttimeout() does not bound socket.gethostbyname() name-resolution time. In that environment, every model load that enters _hf_offline_if_dns_dead() can hang before it reaches the cached/offline fallback, so use a resolver strategy with an actual deadline instead of relying on the socket default timeout.

Useful? React with 👍 / 👎.

Keep offline scopes isolated per request

When two load requests overlap while DNS is down, the second one can enter here while the first request's context has temporarily set HF_HUB_OFFLINE; it then yields without owning/restoring the setting, and the first request's finally later pops the variable while the second request is still running. Because environment variables are process-wide, that second request can unexpectedly fall back to normal HF network calls/timeouts mid-scope; guard this with a lock/refcount or avoid using process-wide env mutation for per-request state.

Useful? React with 👍 / 👎.

Honor ON in offline environment flags

When users set HF_HUB_OFFLINE=ON or TRANSFORMERS_OFFLINE=ON (standard truthy values for Hugging Face/Transformers), this helper returns False, so the new offline short-circuits below are skipped and _check_tokenizer_config_needs_v5 / _check_config_needs_550 still perform raw urllib fetches that can wait up to 10 seconds each before failing. This undermines the offline path added here for a supported env spelling; include on in the truthy set, and keep the duplicate offline helper consistent.

Useful? React with 👍 / 👎.

Avoid mutating process-wide socket defaults in DNS probe

_probe_dns_dead() changes the global socket default timeout via socket.setdefaulttimeout(timeout) before resolving huggingface.co. Because this helper is used on the FastAPI parent load path, concurrent requests in the same process can create sockets during this window and unexpectedly inherit the 2s default, causing unrelated outbound calls to fail or time out early. Please switch to a probe mechanism that enforces its own deadline without touching process-wide socket defaults.

Useful? React with 👍 / 👎.

Reuse a bounded DNS probe thread

When DNS resolution hangs (the _probe thread never returns), join(timeout) exits but the daemon thread keeps running, and each call to this helper spawns another stuck thread. Because _hf_offline_if_dns_dead() is now used on model-load paths in the long-lived inference parent process, repeated loads during a resolver outage can accumulate blocked threads and eventually hit thread/memory limits, degrading or breaking further requests.

Useful? React with 👍 / 👎.

Treat HF_HUB_OFFLINE as enabled only when truthy

This guard checks only for variable presence, so HF_HUB_OFFLINE=0 (a common explicit default) disables the DNS auto-detect path entirely. In that configuration, dead-DNS loads bypass the new offline short-circuit and fall back to slow network timeouts again, which defeats the regression fix this change is introducing.

Useful? React with 👍 / 👎.

Run DNS probe when HF_HUB_OFFLINE is falsy

This guard treats any presence of HF_HUB_OFFLINE as an opt-out, so common explicit defaults like HF_HUB_OFFLINE=0 skip the auto-detect path entirely. In that configuration, a dead resolver no longer flips the worker to offline mode and training falls back to slow Hugging Face/network timeouts, which defeats the latency regression fix this block is meant to provide for cached models.

Useful? React with 👍 / 👎.

Read cached configs before defaulting offline

When HF_HUB_OFFLINE is set for a cached remote model whose required transformers tier is only discoverable from tokenizer_config.json/config.json (for example a forked Gemma4-style repo without the hard-coded name substring), this new early return skips the cached HF snapshot and forces _check_tokenizer_config_needs_v5 to False; the duplicated guard in _check_config_needs_550 does the same. activate_transformers_for_subprocess() then selects the default 4.57.x tier even though the needed metadata is already on disk, so offline training/inference of those cached models can fail with the wrong transformers version instead of avoiding only the network fetch.

Useful? React with 👍 / 👎.

Don't cache transient offline tier defaults

When _hf_offline_if_dns_dead() temporarily enables offline mode for a parent-process load and this check runs for a remote model without a local config.json, this stores False in the process-wide _config_needs_550_cache. Because the cache is consulted before any later local/network read, the same model remains classified as not needing transformers 5.5.0 even after DNS recovers, so the inference orchestrator/vision detection can keep spawning workers with the wrong transformers tier. The offline shortcut should avoid populating the normal cache, or key the cached value on the offline state.

Useful? React with 👍 / 👎.